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The  Honorable  John  J.  Duncan 
Chairman,  Subcommittee  on  Aviation 
Committee  on  Transportation  and  Infrastructure 
House  of  Representatives 

The  Honorable  Thomas  W.  Ewing 
House  of  Representatives 

In  1998,  general  aviation  had  1,907  accidents,  with  621  fatalities.  The 
National  Transportation  Safety  Board  estimates  that  87  percent  of  all  fatal 
general  aviation  accidents  are  caused  by  pilot  error,  especially  when  pilots 
who  do  not  have  appropriate  instrument  training  fly  when  visibility  is 
poor,  such  as  during  bad  weather.  To  reduce  the  occurrence  of  general 
aviation  accidents,  the  Federal  Aviation  Administration  (faa)  has  been 
exploring  a  number  of  means  to  enhance  the  training  of  general  aviation 
pilots. 

One  possible  enhancement  is  the  use  of  new  technologies  for  the  training 
that  occurs  on  the  ground.  For  over  40  years,  general  aviation  student 
pilots  have  used  flight  training  devices  to  help  them  learn  how  to  fly  using 
an  aircraft’s  instruments  alone.  These  flight-training  devices  resemble  an 
aircraft’s  cockpit  and  are  often  constructed  with  actual  airplane 
instruments;  they  can  be  used  by  student  pilots  to  substitute  for  up  to  20  of 
the  40  hours  of  airplane  training  required  by  faa  to  obtain  an  instrument 
rating.  The  instrument  rating  permits  a  pilot  to  fly  when  visibility  is  poor. 

In  May  1997,  faa  also  approved  the  use  of  special  personal  computers, 
controls,  and  software  called  personal  computer-based  aviation  training 
devices  (pcatd),  which  can  be  used  for  up  to  10  hours  of  instrument 
training.1  faa’s  decision  to  allow  the  use  of  pcatds  has  sparked  debate. 
Some  assert  that  pilots  trained  with  these  devices  will  be  less  skilled, 
thereby  compromising  aviation  safety.  Others  argue  that  pilots  trained 
with  the  devices  are  actually  better  trained  at  lower  cost. 

As  a  result  of  this  controversy,  you  asked  us  to  (1)  describe  the  process 
and  information  faa  used  in  deciding  to  approve  the  use  of  personal 


‘“Personal  Computer-Based  Aviation  Training  Device."  or  PCATD,  is  FAA’s  term  for  computer-based 
devices  that  have  been  approved  for  use,  or  qualified  for  credit  hour  training,  by  FAA  officials.  FAA 
requires  PCATDs  to  be  qualified,  approved,  and  used  in  connection  with  an  integrated  ground  and 
flight  instrument  training  curriculum.  Training  using  PCATDs  should  include  the  procedural  tasks 
listed  in  FAA’s  Advisory  Circular  61-126.  In  this  report,  we  refer  to  these  devices  as  computer-based 
devices  before  they  were  approved  for  use  in  training  and  PCATDs  once  the  devices  were  approved  for 
use  by  FAA.  PCATDs  describes  one  of  the  four  FAA-approved  devices. 
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computer  devices  for  10  hours  of  instrument  training  and  (2)  discuss  what 
is  known  about  the  training  effectiveness  of  these  devices  and  their 
long-term  impact  on  a  pilot’s  ability  to  fly  safely.  To  respond  to  these 
objectives,  we  interviewed  federal  officials  and  others  familiar  with  faa’s 
decision  and  reviewed  faa’s  supporting  documentation;  conducted  a 
comprehensive  literature  search  on  the  use  of  these  and  similar 
computer-based  devices,  on  instrument  training,  and  on  general  aviation 
safety;  reviewed  about  700  studies  and  articles;  analyzed  and  summarized 
the  most  relevant  data-based  literature;  interviewed  government, 
academic,  and  private  sector  flight  instruction  experts  on  the  use  of  the 
devices;  and  identified  other  issues  related  to  the  devices’  potential 
impacts  on  aviation  safety. 


Results  in  Brief 


faa’s  decision  to  allow  the  use  of  computer-based  devices  for  instrument 
flight  training  took  over  6  years  to  be  made  final  and  was  based  on  two 
major  research  studies,  faa’s  professionaljudgment,  and  input  from 
aviation  industry  representatives.  Because  the  two  studies  did  not  address 
the  appropriate  number  of  hours  of  training  on  these  devices,  faa’s 
decision  to  allow  10  hours  was  based  on  its  professionaljudgment  and 
industry  input,  faa  had  earlier  relied  on  its  professionaljudgment,  rather 
than  empirical  studies,  to  support  its  approval  of  up  to  20  credit  hours  for 
the  devices  now  known  as  flight  training  devices.  A  University  of  Illinois 
research  study  is  planned  to  assess  the  appropriate  number  of  credit  hours 
for  the  use  of  these  devices  in  an  instrument  training  course. 

The  two  major  research  studies  generally  support  the  use  of 
computer-based  devices  for  training.  Despite  some  methodological 
limitations,  these  are  the  most  complete  controlled  studies  to  date  on  the 
training  effectiveness  of  computer-based  devices.  One  study  shows  that 
the  use  of  computer-based  devices  may  modestly  reduce  the  training  time 
spent  in  an  airplane,  while  the  other  shows  that  the  training  effects  of 
computer-based  devices  and  previously  approved  flight  training  devices 
may  not  differ  greatly.  Other  studies  we  reviewed  also  generally  supported 
the  use  of  computer-based  devices  in  training.  Similarly,  the  majority  of 
the  experts  we  interviewed  saw  some  training  value  from  the  use  of  the 
devices  and  did  not  believe  that  they  were  likely  to  reduce  aviation  safety. 
Although  most  experts  did  not  speculate  on  the  appropriate  number  of 
credit  hours  that  should  be  granted  for  the  devices,  several  disagreed  with 
faa’s  decision  to  allow  any  credit  hours.  We  found  no  empirical  evidence, 
however,  on  the  long-term  safety  impact  of  the  devices,  their  potential 
safety  benefits,  or  the  long-term  safety  impact  or  benefits  of  the  currently 
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approved  flight  training  devices.  Moreover,  faa  does  not  currently  collect 
the  data  needed  to  conduct  future  research  on  a  possible  link  between  the 
use  of  the  devices  and  pilots’  long-term  safety  records. 

This  report  recommends  that  faa  gather  additional  information  on  the 
long-term  safety  issues  associated  with  computer-based  devices  and 
previously  approved  flight  training  devices,  faa  will  soon  revise  and 
computerize  its  pilot  application  form.  This  form  could  collect  additional 
information  at  minimal  cost  on  the  use  of  pilot  training  devices  and  would 
facilitate  future  research. 


Background 


To  establish  training  requirements  for  pilots,  FAA  issues  regulations  for 
pilots,  pilot  schools,  instructors,  and  pilot  training  equipment 
manufacturers,  and  provides  guidance  through  advisory  circulars, 
handbooks,  and  other  types  of  informational  material.2  Within  faa,  several 
organizations  regulate  pilot  training.  For  example,  the  Flight  Standards 
Office  is  responsible  for  certifying  and  overseeing  pilot  schools,  pilots,  and 
pilot  training  equipment.  Likewise,  faa’s  Civil  Aeromedical  Institute  (cami) 
studies  human  performance  factors  in  aviation,  among  other  issues. 

Finally,  faa’s  National  Simulator  Program  studies  and  evaluates  the  more 
sophisticated  simulator  equipment  used  in  training  pilots. 

Beginning  students  first  learn  to  fly  under  faa’s  visual  flight  rules,  which 
generally  require  a  minimum  visibility  of  3  miles,  although  most  new  pilots 
fly  with  much  better  visibility.  Pilots  then  must  obtain  an  instrument  rating 
to  fly  in  conditions  of  reduced  visibility,  such  as  in  clouds  or  in  poor 
weather.  Flying  when  visibility  is  poor  is  more  difficult  because  a  pilot 
loses  reference  to  the  outside  horizon  and  must  rely  completely  on 
instruments  to  know  the  plane’s  spatial  orientation.  Instrument  training  is 
important  to  aviation  safety  because  the  vast  majority  of  accidents  involve 
poor  weather  and  poor  visibility.  When  a  pilot  without  instrument  training 
mistakenly  flies  into  conditions  of  poor  visibility,  disorientation  and  loss  of 
control  can  occur  quickly;  many  fatal  weather-related  accidents  occur 
when  a  pilot  who  is  flying  according  to  visual  flight  rules  accidentally 
encounters  poor  visibility  conditions  that  are  beyond  his  or  her  abilities. 
According  to  experts,  good  instrument  skills  are  critical  to  flying  safely 
through  bad  weather. 


2The  regulation  of  pilots,  pilot  schools,  and  pilot  training  equipment  by  FAA  is  governed  by  title  14, 
C.F.R.  parts  61,  91, 121, 135,  and  141. 
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An  instrument  rating  is  considered  difficult  to  obtain  and  normally 
requires  additional  training,  an  faa  written  exam,  and  a  practical 
test— called  a  “check  ride”— with  an  FAA-designated  examiner  in  an 
aircraft.  Before  taking  these  exams,  pilots  must  take  an  additional  35  to  40 
hours  of  training,  which  may  include  up  to  20  hours  in  a  flight  simulator  or 
flight  training  device,  with  the  remaining  time  in  an  aircraft.3  However, 
pilots  typically  take  about  70  hours  of  training  in  an  aircraft  or  training 
device  before  they  are  proficient  enough  to  pass  the  tests  needed  to  obtain 
their  instrument  rating. 

In  May  1997,  faa  issued  an  advisory  circular  approving  the  use  of  pcatds 
for  training  credit  toward  the  instrument  rating.4  These  aviation  training 
devices  are  personal  computers  with  specialized  software  and  add-on 
equipment,  including  separate  simulated  radio  and  navigational  controls  to 
the  right  of  the  computer  monitor,  a  throttle,  a  steering  yoke  under  the 
computer  screen,  and  rudder  pedals  on  the  floor.  A  fully  equipped 
FAA-approved  pcatd  costs  from  $5,000  to  $10,000.  In  comparison,  a 
previously  approved  flight  training  device  costs  from  $40,000  to  $90,000.5 
When  training  with  either  a  pcatd  or  a  flight  training  device,  the  instructor 
usually  sits  with  a  student,  sometimes  at  a  separate  personal  computer 
console  where  the  instructor  can  plan,  monitor,  and  control  a  student’s 
flight.  Figure  1  shows  an  FAA-approved  pcatd. 


3There  are  two  types  of  pilot  training  schools.  One  is  an  FAA-approved  school  (Part  141)  and  others  are 
nonapproved  schools  that  still  must  meet  FAA  training  requirements  (Part  61).  An  FAA-approved  school 
may  be  authorized  to  give  its  graduates  written  exams  and  check  rides.  In  addition,  an  FAA-approved 
school  can  qualify  pilots  with  fewer  hours  than  can  a  nonapproved  school.  Many  of  the  nonapproved 
pilot  schools  find  it  impractical  to  qualify  for  FAA  certificates,  but  85  percent  of  all  instrument  ratings 
are  granted  for  the  training  conducted  at  these  schools. 

4FAA  Advisory  Circular  61-126. 

5In  contrast,  a  full-motion  simulator  used  by  airlines  to  train  commercial  pilots  can  cost  up  to 
$20  million. 
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Figure  1:  Personal  Computer-Based 
Aviation  Training  Device 


Source:  Aviation  Supplies  and  Academics,  Inc. 
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faa’s  advisory  circular  spells  out  the  conditions  that  an  aviation  training 
device  must  meet  in  order  to  be  qualified  by  faa  as  a  pcatd  for  use  in 
satisfying  the  instrument  rating  requirements.  The  advisory  circular’s 
guidelines  are  not  mandatory  and  contain  a  voluntary  reporting 
mechanism  requesting  information  on  students’  pcatd  use.  faa  could  have 
used  its  rule-making  authority  to  approve  pcatd  use.  It  did  not  do  so  for 
several  reasons,  according  to  Flight  Standards  officials.  First,  rules  are 
mandatory  and  advisory  circulars  are  not,  making  any  significant  changes 
to  mandatory  rules  more  difficult  and  time-consuming  than  making  any 
significant  changes  to  advisory  circulars.  Second,  in  the  past,  the 
qualification  of  ground  flight  training  devices  in  general  aviation  was 
handled  by  issuing  advisory  circulars.  Finally,  faa  officials  wanted  to 
facilitate  the  rapid  introduction  of  the  new  pcatd  technology  without  going 
through  a  lengthy  rule-making  process.  Once  faa  has  qualified  a 
pcatd — including  its  hardware,  software,  and  accompanying 
equipment — the  device  can  be  used  by  a  flight  school  or  an  individual 
flight  instructor  in  an  integrated  ground  and  flight  training  curriculum  for 
up  to  10  hours  of  flight  instruction  toward  an  instrument  ;rating. 


FAA’s  Decision  to  Allow 
the  Use  of  Computer-Based 
Devices  Was  Based  on  Two 
Studies,  FAA’s  Professional 
Judgment,  and  Industry 
Input 


faa’s  decision  to  allow  the  use  of  computer-based  devices  and  to  grant 
credit  hours  evolved  over  many  years.  Since  the  1980s,  faa  had  been 
evaluating  various  designs  of  pcatds  but  did  not  consider  these  early 
devices  to  be  sophisticated  enough  for  general  use  in  instrument  flight 
courses.  As  computer  equipment  advanced  and  training  software  was 
refined,  faa  in  1991  began  to  seriously  consider  proposals  from 
manufacturers  and  aviation  schools  to  allow  the  use  of  computer-based 
devices  for  instrument  flight  training.  Initially,  faa  officials  considered 
allowing  the  use  of  the  devices  for  general  instructional  purposes  only  and 
not  for  students  to  earn  flight  hours  toward  an  instrument  rating. 


In  1992,  Flight  Standards  officials  proposed  developing  a  research  project 
to  obtain  information  on  the  use  of  the  devices  because  they  were 
concerned  that  they  needed  empirical  data  to  support  the  introduction  of 
this  new  training  technology.  According  to  Flight  Standards  officials,  faa 
has  never  sponsored  a  study  that  evaluated  the  performance  of  previously 
approved  flight  training  devices,  which  remain  in  wide  use.  faa  relied  on 
its  professional  judgment  rather  than  empirical  studies  to  support  the 
approval  of  up  to  20  credit  hours  for  flight  training  devices. 

Later  in  1992,  faa  provided  a  grant  to  Embry-Riddle  Aeronautical 
University  to  study  the  use  of  computer-based  devices.  A  principle 
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objective  of  the  study  was  to  measure  the  transfer  of  instrument  training 
knowledge  to  an  aircraft  for  three  groups  of  students  trained  using  two 
different  computer-based  training  devices  and  one  approved  flight  training 
device.  According  to  Flight  Standards  officials,  the  Embry-Riddle  study 
was  designed  to  provide  them  with  information  on  whether  the  devices 
were  useful  and  effective.  In  June  1993,  while  the  Embry-Riddle  research 
was  ongoing,  cami’s  Human  Factors  Research  Laboratory  submitted  a 
research  proposal  to  expand  and  participate  in  the  study.  This  detailed 
approach  was  not  approved,  according  to  a  Flight  Standards  official, 
because  faa  at  the  time  did  not  have  the  funds  to  support  it.  For  the  next 
year,  cami  officials  continued  to  help  explore  possible  research 
methodologies  for  qualifying  the  devices  in  a  structured  training 
curriculum  that  would  provide  some  insight  into  determining  if  credit 
hours  should  be  granted  and  what  tasks  the  devices  are  best  suited  for  in 
instrument  training.  However,  at  that  time,  faa  was  not  proposing  to  grant 
flight  training  credit  hours  for  training  on  computer-based  devices. 

When  the  Embry-Riddle  study  was  completed  in  1994,  some  faa  officials 
outside  of  Flight  Standards  believed  that  the  study  had  significant 
limitations,  was  not  properly  designed,  and  did  not  fully  answer  important 
questions  on  whether  computer-based  devices  should  be  approved,  cami 
officials  stated  that  the  failure  to  include  a  control  group  that  received 
training  only  in  the  aircraft  seriously  limited  the  usefulness  of  the  study.  In 
addition,  officials  in  faa’s  National  Simulator  Program  stated  that  no 
meaningful  conclusion  could  be  drawn  from  the  research. 

In  June  1994,  faa  sponsored  a  meeting  of  manufacturers  and  others  in  the 
aviation  training  industry  to  obtain  their  views  on  faa’s  current  and  future 
direction  in  approving  the  use  of  computer-based  devices.  In  addition, 
starting  in  1994,  FAA  sponsored  further  research  through  a  series  of  grants 
to  the  University  of  Illinois  to  study  the  training  effectiveness  of  the 
devices  and  to  answer  many  of  the  questions  and  concerns  about  their  use 
that  remained  following  the  Embry-Riddle  study. 

While  faa  was  evaluating  the  qualifications  of  computer-based  devices  for 
instrument  training  without  credit  hours,  it  received  requests  to  allow 
some  of  the  training  hours  done  on  the  equipment  to  be  credited  toward 
the  minimum  hours  of  flight  time  in  instrument  training.  Some 
manufacturers,  for  example,  wanted  faa  to  consider  granting  credit  hours 
in  order  to  make  their  equipment  more  credible  to  their  customers. 
Furthermore,  in  May  1995,  the  Aircraft  Owners  and  Pilots  Association’s  Air 
Safety  Foundation  formally  petitioned  faa  to  allow  training  on 
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computer-based  devices  to  count  for  training  time  toward  an  instrument 
rating  and  proposed  a  syllabus  illustrating  the  integration  of  the  devices 
into  an  instrument  flight  course. 

Although  neither  the  Embry-Riddle  nor  the  University  of  Illinois  study 
addressed  any  appropriate  number  of  flight  training  credit  hours,  FAA 
officials  used  their  experience  and  professional  judgment  to  support  their 
decision  and  saw  it  as  a  reasonable  compromise  between  the  various 
proposals.  Furthermore,  faa  officials  decided  to  add  a  voluntary  reporting 
mechanism  to  the  draft  advisory  circular  designed  to  gather  information  to 
validate  the  approval  and  use  of  the  devices  with  feedback  from  users. 
Finally,  faa  officials  told  us  that  the  required  check  ride  by  an 
FAA-designated  examiner  after  a  student’s  training  would  ensure  that  all 
instrument  students  were  qualified  to  receive  the  instrument  rating,  no 
matter  what  equipment  they  used  during  training.  Several  flight  training 
experts  we  interviewed,  however,  told  us  that  the  check  ride  is  an 
imperfect  measure  of  a  pilot’s  ability,  should  not  be  seen  as  assurance  of 
proper  training,  and  is  instead  a  snapshot  of  a  pilot  on  a  single  day.  These 
experts  believe  assurances  on  the  quality  of  training  devices,  a  quality 
curriculum,  capable  instructors,  consistent  testing  by  FAA-designated  flight 
examiners,  and  long-term  research  on  pilot  safety  and  training  are  needed. 

In  August  1995,  FAA  sponsored  an  additional  meeting  of  aviation  industry 
officials  and  presented  a  draft  advisory  circular  that  proposed,  for  the  first 
time,  granting  10  flight  training  credit  hours.  The  draft  advisory  circular 
and  all  significant  changes  to  the  draft  were  submitted  to  over  60  groups, 
organizations,  and  other  interested  parties.  For  the  next  12  months,  faa 
officials  received  numerous  comments  on  the  use  of  computer-based 
devices.  During  this  time,  several  aviation  industry  officials  suggested  that 
faa  delay  or  withhold  approval  of  a  final  advisory  circular  pending  more 
research  and  the  completion  of  the  University  of  Illinois  study.  In 
September  1996,  faa  decided  to  delay  approving  the  advisory  circular  until 
after  the  University  of  Illinois  study  was  completed. 

In  November  1996,  the  University  of  Illinois  issued  its  final  report  and  in 
March  1997  briefed  faa  officials  on  the  results.  The  study  concluded,  in 
part,  that  computer-based  devices  can  provide  a  worthwhile  training 
benefit  and  can  save  a  small  portion  of  the  aircraft  time  that  would 
otherwise  be  needed.  In  January  1997,  the  National  Air  Transportation 
Association,  after  extensive  discussion,  recommended  that  faa  approve  a 
draft  advisory  circular,  including  flight  training  credit  hours  for 
computer-based  devices,  as  well  as  other  recommendations. 
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In  May  1997,  faa  issued  Advisory  Circular  61-126,  which  created  a  separate 
category  of  training  device — pcatd — rather  than  integrating  pcatds  into  the 
“ground  training  device”  category  already  approved  for  credit  hours  in 
instrument  training.  These  existing  ground  training  devices  were  renamed 
flight  training  devices  (ftd)  to  distinguish  them  from  pcatds.  Older  models 
of  the  new  ftd  category  were  grandfathered  in  and  automatically  approved 
by  faa  for  instrument  credit  hours.  According  to  Flight  Standards  officials, 
the  decision  to  approve  10  hours  of  flight  credit  was  based  on  (1)  the 
general  utility  of  computer-based  devices  in  instrument  training,  as  shown 
by  the  Embry-Riddle  and  University  of  Illinois  studies;  (2)  a  wide  variety  of 
proposals,  ranging  from  zero  hours  of  credit  to  20  hours;  and  (3)  the  input 
from  manufacturers  that  they  needed  credit  hours  to  sell  their  equipment. 
Two  additional  research  studies  are  planned:  One  will  try  to  assess  the 
appropriate  number  of  credit  hours  for  pcatd  use  in  an  instrument  course; 
the  second  will  investigate  the  use  of  pcatds  to  maintain  instrument  skills.6 


Over  the  next  several  months,  FAA  officials  qualified  four  pcatd  devices, 
which  included  three  software  programs  and  several  other  manufacturers 
of  controls,  while  finding  an  equal  number  unacceptable.  As  of  June  1999, 
no  information  had  been  sent  to  faa  under  the  advisory  circular’s 
voluntary  reporting  mechanism  on  the  use  of  pcatds.  An  official  of  the 
National  Association  of  Flight  Instructors  estimated  that  very  few  of  the 
200  Part  141  flight  schools  now  use  pcatds  for  instrument  instruction. 


6This  University  of  Illinois  research  study  is  likely  to  be  funded  by  the  National  Aeronautics  and  Space 
Administration’s  Ames  Research  Laboratory,  which  investigates  human  factors  issues  and  funds 
research  investigating  general  and  commercial  aviation  safety.  The  study  proposes  to  investigate,  over 
3  years,  the  progress  of  three  groups  of  instrument  students  using  PCATDs  for  5, 10,  and  15  hours  of 
training  in  an  instrument  training  course.  A  control  group  would  receive  all  training  in  an  airplane.  The 
use  of  flight  training  devices  will  not  be  included  in  the  study.  An  additional  study  is  now  examining 
the  training  value  of  PCATDs  for  maintaining  pilots’  skills  and  for  substituting  these  devices  for 
aircraft  experience  in  maintaining  instrument  skills.  According  to  a  University  of  Illinois  researcher, 
the  airplane  performance  of  100  current  instrument  pilots  will  be  tested  and  compared  across  four 
groups:  A  control  group  will  receive  no  practice  in  instrument  flight  during  a  6-month  period;  an 
airplane  group  will  perform  the  six  required  approaches  and  holding  patterns  in  the  aircraft;  a  PCATD 
group  will  perform  the  same  maneuvers  in  a  PCATD;  and  a  flight  training  device  group  will  perform 
the  same  maneuvers  in  a  flight  training  device.  The  performance  of  pilots  on  the  final  instrument 
proficiency  check  ride  at  the  end  of  the  6-month  experimental  period  will  be  compared  with  the 
performance  on  the  initial  instrument  proficiencies  check  ride.  Pilots  may  not  receive  recency  credit 
for  their  practice  on  PCATDs.  To  meet  FAA  recency  regulations  toward  maintaining  current 
instrument  skills,  pilots  must  make  at  least  six  instrument  approaches,  holding  procedures,  and 
intercepting  and  tracking  courses  every  6  months.  These  maneuvers  must  be  performed  under  either 
simulated  or  actual  flight  conditions;  currently,  flight  training  devices  may  be  used  to  perform  all  six 
instrument  currency  approaches,  but  PCATDs  may  not. 
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Research  and  Experts 
Support  Training 
Value  of  PCATDs,  but 
Devices’  Impact  on 
Long-Term  Safety  Is 
Unknown 


The  two  FAA-commissioned  studies  by  the  University  of  Illinois  and 
Embry-Riddle  Aeronautical  University  are  the  most  complete  controlled 
studies  to  date  on  the  training  effectiveness  of  computer-based  devices. 
Despite  methodological  limitations,  the  studies  show  that  (1)  the  use  of 
computer-based  devices  may  modestly  reduce  the  training  time  spent  in 
the  airplane  (Illinois)  and  (2)  the  training  effects  of  computer-based 
devices  and  a  previously  approved  flight  training  device  may  not  differ 
greatly  (Embry-Riddle).  The  additional  studies  we  reviewed  also 
supported  the  use  of  computer-based  devices.  None  of  these  studies 
examined  the  issue  of  how  many  hours  of  training  on  a  computer-based 
device  should  be  allowed  in  an  instrument  training  curriculum.  Academic 
and  other  flight  training  experts  generally  believed  that  computer-based 
devices  offer  training  value  to  pilots  obtaining  an  instrument  rating. 


Since  pcatds  have  only  recently  been  approved  and  few  pilots  have  used 
them  in  training,  their  long-term  risks  and  benefits  are  not  yet  known. 
While  several  studies  examined  the  training  effectiveness  of 
computer-based  devices,  none  directly  addressed  their  effects  on  general 
aviation  safety.  Safety  experts  we  interviewed  did  not  believe  that  the  use 
of  computer-based  devices  and  the  granting  of  credit  hours  pose  a  threat 
to  general  aviation  safety  by  adversely  affecting  the  skills  and  abilities  of 
pilots  who  receive  instrument  training  using  the  devices.  However,  several 
other  issues  that  could  affect  the  long-term  ability  of  PCATD-trained  pilots 
to  fly  safely  have  been  raised. 


Research  Generally 
Supports  Use  of  PCATDs  in 
Training 


The  Illinois  study  showed  that  the  use  of  computer-based  devices  in  an 
instrument  course  may  modestly  reduce  the  training  time  spent  in  the 
airplane — 26.5  hours  on  the  device  during  an  instrument  course  reduced 
the  amount  of  airplane  training  time  needed  by  3.9  hours.  (See.  fig.  2.) 
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Figure  2:  University  of  Illinois  Study: 
Hours  to  Proficiency  in  an  Airplane 


Hours 


Airplane  Only 


One  group  of  students  One  group  of  students 
trained  only  in  an  airplane,  trained  using  a  personal 

computer  and  in  an  airplane. 


|  |  Hours  using  PC 


1  Hours  in  airplane 


Notes:  N  =  Number  of  students  trained. 

The  airplane  used  was  a  Beechcraft  Sports/Sundowner.  The  computer-based  device  was  an 
MDM  FS-100,  not  FAA-approved  as  a  PCATD. 

Source:  GAO’s  analysis  of  University  of  Illinois  data. 


The  Embry-Riddle  study  provided  a  preliminary  indication  that  there  may 
not  be  very  large  differences  between  the  training  effects  of  the  previously 
approved  flight  training  devices  and  the  recently  approved 
computer-based  devices.  The  training  trials  and  time  needed  to  reach 
proficiency  for  students  using  two  different  types  of  computer-based 
devices  and  for  students  using  a  previously  approved  flight  training  device 
are  shown  in  figures  3  and  4.  (See  app.  II  for  a  more  detailed  summary  of 
the  two  FAA-sponsored  studies.) 
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Figure  3:  Embry-Riddie  Study:  Trials  to  ■■ 
Proficiency  in  an  Airplane  Trials 


Elite  PC  FT  PC  Frasca  FTD 


Training  device  used 


Notes:  N  =  Number  of  students  trained. 

The  airplane  used  was  the  Mooney  20J.  The  Elite  PC  was  a  computer-based  device,  an 
FAA-approved  PCATD.  The  IFT  PC  was  a  computer-based  device  not  approved  by  FAA  as  a 
PCATD.  The  Frasca  FTD  is  a  flight-training  device  approved  by  FAA. 

Source:  GAO’s  analysis  of  Embry-Riddie  Aeronautical  University  data. 
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Figure  4:  Embry-Riddle  Study:  Hours 
to  Proficiency  in  an  Airplane 


Hours 


Elite  PC  IFT  PC  Frasca  FTD 


Training  device  used 


Notes:  N  =  Number  of  students  trained. 

The  airplane  used  was  the  Mooney  20J.  The  Elite  PC  was  a  computer-based  device,  an 
FAA-approved  PCATD.  The  IFT  PC  was  a  computer-based  device  not  approved  by  FAA  as  a 
PCATD.  The  Frasca  FTD  is  a  flight-training  device  approved  by  FAA. 

Source:  GAO’s  analysis  of  Embry-Riddle  Aeronautical  University  data. 


The  additional  studies  we  reviewed  also  found  some  value  in  using 
computer-based  devices.  Three  studies  found  computer-based  devices 
helped  train  beginning  pilots.7  An  additional  study  found  that  performance 
in  an  airplane  test  was  better  for  a  group  of  students  trained  on 


7Gustavo  Ortiz,  “Effectiveness  of  PC-Based  Flight  Simulation,”  The  International  Journal  of 
Aviation  Psychology  4  (3),  p.  285, 1994;  Kerry  A  Dennis  and  Don  Harris,  “Computer-Based 
Simulation  as  an  Adjunct  to  Ab  Initio  Flight  Training,”  The  International  Journal  of  Aviation 
Psychology  8  (3),  pp.  261-276, 1998;  Jefferson  M.  Koonce,  Steven  L.  Moore,  and  Charles  J.  Benton, 
“Initial  Validation  of  a  Basic  Flight  Instruction  Tutoring  System  (BFITS),”  Proceedings  of  the  Eighth 
International  Symposium  on  Aviation  Psychology,  Columbus,  Ohio,  1995. 


Page  13 


GAO/RCED-99-143  Personal  Computer-Based  Aviation  Training  Device 


computer-based  devices  than  for  a  group  trained  on  flight  training  devices.8 


Experts  See  Training  Value 
in  Computer-Based 
Devices 


The  academic  and  flight  training  experts  as  well  as  the  flight  instructors 
we  interviewed  generally  believed  that  pcatds  offer  training  value  to 
instrument  students.  Experts  told  us  that  training  devices,  if  used  properly, 
have  the  potential  to  be  more  effective  than  airplane  training  for  certain 
uses;  furthermore,  training  experts  generally  agree  that  a  low-cost 
simulation  device  may  be  justified  economically  even  if  the  training  it 
provides  is  less  effective  than  the  training  provided  in  an  airplane.  In 
addition,  commercial  airline  training  and  military  research  support  the 
value  of  computer-based  training  for  certain  training  purposes.  Several 
experts  we  interviewed  believed  that  computer-based  devices  would 
become  more  effective  as  their  technology  continues  to  improve  and  as 
features  are  added  to  enhance  their  training  effectiveness. 


Some  experts  believe  that  a  variety  of  training  equipment,  including 
computer-based  devices,  can  offer  effective  training  if  used  properly. 
pcatds  are  usually  seen  by  experts  as  most  appropriate  for  introducing 
equipment  and  procedures  and  for  practicing  certain  maneuvers,  flight 
scenarios,  and  emergency  procedures.  Flight  instructors  in  one  survey 
believed  that  computer-based  devices  were  effective  for  teaching  most 
tasks  in  an  instrument  course.9 


Flight  training  experts  at  a  major  airline’s  training  center  told  us  that  they 
and  other  airlines  use  a  variety  of  training  equipment  in  their  operations. 


8Sybil  I.  Phillips,  Charles  L.  Hulin,  and  Paul  J.  Lamermayer,  “Uses  of  Part-Task  Trainers  in  Instrument 
Flight  Training,”  University  of  Illinois,  Proceedings  of  the  Seventh  International  Symposium  on 
Aviation  Psychology,  Columbus,  Ohio,  1993. 

9WiIliam  F.  Moroney,  Steven  Hampton,  and  David  W.  Biers,  “Considerations  in  the  Design  and  Use  of 
Personal  Computer-Based  Aircraft  Training  Devices  (PCATDs)  for  Instrument  Flight  Training:  A 
Survey  of  Instructors,”  University  of  Dayton,  Ohio,  and  Embry-Riddle  Aeronautical  University, 
Daytona  Beach,  Florida,  Proceedings  of  the  Ninth  International  Symposium  on  Aviation  Psychology, 
Columbus,  Ohio,  1997.  According  to  another  study  (Brian  K.  Rogers,  Capt.,  U.S.  Air  Force. 
“Microcomputer-Based  Instrument  Flight  Simulation:  Undergraduate  Pilot  Training  Student  Attitude 
Assessment,”  December  1991,  Human  Resources  Directorate,  Aircrew  Training  Research  Division, 
Williams  Air  Force  Base,  Arizona,  AL-TR-1 991 -0039),  surveyed  students  believe  that  computer-based 
devices  have  some  training  value.  Almost  all  of  the  of  U.S.  Air  Force  flight  students  surveyed  planned 
to  use  computer-based  instrument  simulators  to  replace  at  least  some  of  their  “chair-flying,”  or 
rehearsing  procedures  from  a  chair  placed  in  front  of  a  life-size  photo  of  the  airplane’s  cockpit.  Over 
half  said  they  would  replace  at  least  75  percent  of  their  chair-flying  with  computer-based  simulation. 
The  same  study  found  increased  student  interest  in  using  computer-based  simulation  as  instrument 
training  was  expanded  to  include  multiple  procedures  in  real-time  practice.  Another  study  (D.  Gopher, 
M.  Weil,  and  T.  Bareket,  “Transfer  of  skill  from  a  computer  game  trainer  to  flight,”  Human  Factors,  36 
(3)  (1994),  pp.  387-405)  found  that  military  flight  students  who  played  a  complicated  computer  game 
performed  better  in  later  airplane  training. 
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ranging  from  $20  million  full-motion  simulators  to  simple  personal 
computer-based  training,  depending  on  whether  their  goal  is  to  introduce  a 
pilot  to  a  new  procedure  or  piece  of  equipment,  to  practice  complete 
simulated  flights,  or  to  serve  an  intermediate  purpose.  For  example,  these 
airline  training  officials  believed  that  simple  personal  computer  equipment 
can  introduce  a  pilot  to  a  new  navigation  system  or  component  more 
effectively  than  a  full-motion  simulator  with  its  many  competing  demands 
for  a  pilot’s  attention. 


Safety  and  Academic 
Experts  Generally  Do  Not 
Believe  PCATDs  Pose  a 
Safety  Risk 


According  to  safety  experts  from  the  National  Transportation  Safety  Board 
and  from  an  industry  safety  association,  the  use  of  computer-based 
devices  and  the  granting  of  credit  hours  do  not  pose  a  threat  to  general 
aviation  safety  by  adversely  affecting  the  skills  and  abilities  of  pilots  who 
receive  instrument  training  using  the  devices.  In  general,  academic  and 
other  flight  training  experts  believe  that  the  devices  offer  training  value  to 
pilots  who  want  to  obtain  an  instrument  rating.  According  to  these 
experts,  good  instrument  and  navigational  skills  are  vital  in  avoiding  bad 
weather  and  surviving  it  when  necessary;  two  experts  we  interviewed 
believe  that  the  low  cost  and  wide  availability  of  pcatds  could  help 
maintain  or  increase  the  general  level  of  pilots’  instrument  skills,  which 
are  critical  to  safe  flying. 


Long-Term  Safety  Impact  We  found  no  empirical  evidence  on  the  long-term  safety  impact  of 
of  PCATDs  Is  Unknown  computer-based  devices,  their  potential  safety  benefits,  or  the  long-term 

safety  impact  or  benefits  of  the  currently  approved  flight  training  devices. 
Many  of  the  experts  we  interviewed  believed  either  that  computer-based 
devices  were  not  likely  to  reduce  aviation  safety  or  that  they  offer 
long-term  safety  benefits.  However,  several  experts  noted  that  the 
potential  for  long-term  safety  risks  from  computer-based  devices  used  in 
training  remains  a  possibility.  Objections  to  their  use  for  credit  towards 
the  training  hours  needed  for  an  instrument  rating  generally  revolve 
around  whether  using  a  computer-based  device  in  instrument  training  is 
more  likely  to  reduce  or  increase  an  instrument  pilot’s  margin  of  safety. 
Several  issues  were  raised  in  our  interviews  with  experts  concerning  pcatd 
safety.  Among  these  were  potential  risks  from  the  lack  of  physical 
similarity  between  the  device  and  the  airplane  and  associated  issues, 
possible  benefits  to  training  from  their  use,  and  the  lack  of  research 
showing  that  one  hour  on  a  training  device  equals  one  hour  of  training  in 
an  airplane.  Some  of  these  issues  appear  to  support  the  value  of  pcatds; 
others  raise  questions  about  their  value;  and  the  effects  of  other  issues  on 
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the  value  of  pcatds  are  unclear.  Although  not  fully  resolved,  these  issues 
may  be  relevant  to  future  research.  (See  app.  Ill  for  a  summary  of  these 
issues.) 

faa  does  not  currently  track  how  many  hours  an  instrument  pilot  trains  on 
a  pcatd.  However,  FAA  is  now  computerizing  this  information  and,  for 
unrelated  reasons,  plans  to  revise  the  form  on  which  pilots  apply  for  the 
instrument  and  other  ratings,  faa’s  application  for  the  instrument  rating 
could  easily  and  inexpensively  be  changed  to  include  a  box  on  the  types  of 
training  devices  a  pilot  has  used,  particularly  since  the  form  (faa  Form 
8710-1)  will  soon  be  revised  and  its  data  computerized.  With  information 
on  the  training  devices  used  by  a  particular  pilot,  along  with  other  data 
that  faa  now  maintains  or  will  soon  maintain  electronically,  researchers 
will  be  better  able  to  determine,  within  a  few  years,  the  safety  of  pilots 
trained  using  computer-based  devices.  These  data  will  enable  researchers 
to  link  accidents  or  incidents  with  the  type  of  instrument  training  the 
pilots  had  received.  While  this  type  of  examination  would  present  some 
statistical  challenges,  experts  we  interviewed  said  that  it  would  be 
possible  to  identify  any  large  safety  risks  posed  by  computer-based 
devices. 


Conclusions 


Commercial  aviation  safety  depends  on  many  factors,  including  the 
training  of  general  aviation  pilots,  who  sometimes  share  common  airspace 
and  airports  with  the  flying  public.  Because  most  accidents  in  general 
aviation  are  caused  by  pilot  error,  and  many  accidents  occur  when  pilots 
without  an  instrument  rating  encounter  bad  weather  or  poor  visibility  and 
unexpectedly  have  to  rely  on  their  instruments,  effective  training  that 
teaches,  reinforces,  and  maintains  safe  instrument  flying  skills  is  vital.  To 
the  extent  that  the  use  of  computer-based  devices  may  be  of  training  value 
to  instrument  pilots — which  research  supports  and  experts  believe — the 
devices  do  not  pose  identifiable  safety  risks. 

The  low  cost  and  wide  availability  of  computer-based  devices  could 
improve  the  general  level  of  instrument  training  among  pilots,  according  to 
some  experts,  and  help  to  maintain  or  improve  instrument  pilots’  skills, 
especially  for  students  and  pilots  who  would  not  otherwise  have  access  to 
a  training  device.  Several  experts  told  us  that  faa’s  check  rides,  which  are 
required  for  a  pilot  to  obtain  an  instrument  rating,  may  not  be  sufficient 
indicators  of  a  student’s  training,  capabilities,  and  ability  to  fly  safely. 
However,  we  found  no  studies  on  the  link  between  a  pilot’s  training 
equipment  or  flying  proficiency  at  the  time  of  training  and  that  pilot’s 
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long-term  ability  to  fly  safely.  Collecting  information  on  pilots’  training 
equipment  and  safety  records  would  permit  research  in  the  future  and 
could  help  to  uncover  any  safety  problems  related  to  the  use  of 
computer-based  and  other  training  devices. 

Recommendations 

As  a  first  step  toward  determining  whether  computer-based  devices  pose 
safety  risks  to  general  aviation  in  relation  to  other  training  methods,  the 
Secretary  of  Transportation  should  direct  the  faa  Administrator  to  collect 
information  from  pilot  applications  for  instrument  ratings  on  how  many 
hours  students  trained  on  pcatds  and  flight  training  devices.  This 
information  can  be  used  by  FAA  and  others  to  study  the  relationship 
between  the  use  of  training  devices  and  safety,  faa’s  revision  to  its  rating 
application  form  allows  the  additional  information  to  be  obtained  at 
minimal  cost. 

Agency  Comments 

■ 

We  provided  a  draft  copy  of  this  report  to  the  Office  of  the  Secretary  of 
Transportation  and  the  Federal  Aviation  Administration  (faa)  for  review 
and  comment.  We  discussed  the  report  with  faa  officials,  including  the 
Deputy  Associate  Administrator  for  Regulation  and  Certification.  They 
generally  agreed  with  the  draft  report,  faa  officials  also  indicated  that  they 
concurred  with  and  would  implement  our  recommendation  as  soon  as 
possible.  They  also  provided  technical  corrections,  which  we  incorporated 
into  the  report  as  appropriate. 

Scope  and 
Methodology 

We  reviewed  faa’s  decision  to  allow  the  use  of  pcatds  by  reviewing  the 
data  the  agency  used  to  support  its  decision  and  interviewing  a  variety  of 
faa  and  aviation  industry  officials.  We  also  reviewed  research  on  the  use 
of  computer-based  devices  in  training  and  interviewed  a  variety  of  experts 
on  the  potential  safety  impacts  of  their  use.  Appendix  I  provides  our 
detailed  scope  and  methodology. 

We  conducted  our  review  from  June  1998  through  June  1999  in 
accordance  with  generally  accepted  government  auditing  standards. 

As  arranged  with  your  offices,  unless  you  announce  its  contents  earlier, 
we  plan  no  further  distribution  of  this  report  until  7  days  after  the  date  of 
this  report.  At  that  time,  we  will  make  copies  available  to  the  Secretary  of 
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Transportation  and  the  Administrator,  faa.  We  will  also  make  copies 
available  to  others  on  request. 

If  you  have  any  questions  about  this  report,  please  contact  me  at 

(202)  512-3650.  Major  contributors  to  this  report  were  Karen  Bracey,  David 

Ehrlich,  and  John  Rose. 


Gerald  L.  Dillingham 

Associate  Director,  Transportation  Issues 
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Scope  and  Methodology 


To  examine  the  Federal  Aviation  Administration's  (faa)  decision  allowing 
the  use  of  personal  computer-based  aviation  training  devices  (pcatd),  we 
reviewed  the  data  the  agency  used  to  support  its  decision  to  issue 
Advisory  Circular  61-126,  reviewed  other  studies  available  on  the  topic, 
and  interviewed  a  variety  of  experts  within  and  outside  government.  In 
particular,  we  reviewed  the  supporting  data  faa  used  in  making  its 
decision,  including  a  1994  study  by  Embry-Riddle  Aeronautical  University 
and  a  1996  study  by  the  University  of  Illinois  on  the  effectiveness  of  pcatd 
training.  We  reviewed  faa’s  files,  including  archived  e-mail  messages,  a 
chronology  of  events,  and  supporting  information  and  documents.  We  met 
with  FAA  officials  from  Flight  Standards,  both  in  Washington,  D.C.,  and  at 
an  Illinois  Flight  Standards  District  Office,  as  well  as  with  faa  officials 
from  Human  Factors,  the  Office  of  Aviation  Medicine,  the  Civil 
Aeromedical  Institute  (cami)  in  Oklahoma  City,  and  the  National  Simulator 
Program  in  Atlanta. 

We  asked  several  recognized  academic  and  training  experts  to  review  our 
plans  at  several  points  during  our  research.  Each  of  these  experts  has 
extensive  academic,  research,  and  training  experience  in  aviation, 
including  the  fields  of  aviation  psychology  and  human  factors;  their  views 
are  theirs  alone  rather  than  those  of  their  institutions.  These  experts  were 
Dr.  Barry  Strauch  of  the  National  Transportation  Safety  Board;  Dr.  Dee 
Andrews  of  the  U.S.  Air  Force  Research  Laboratory,  Mesa,  Arizona;  and 
Professor  (Emeritus)  Stanley  Roscoe  of  the  University  of  Illinois  at 
Urbana-Champaign  and  New  Mexico  State  University. 

In  addition,  we  interviewed  flight  training,  aviation  psychology,  human 
factors,  military  research,  and  aviation  safety  experts  in  academia  and  the 
military,  as  well  as  at  the  National  Transportation  Safety  Board  and  the 
National  Aeronautics  and  Space  Administration.  We  interviewed  flight 
instructors,  association  and  airline  officials,  the  manufacturers  of  each  of 
the  approved  pcatds,  and  the  manufacturers  of  flight  training  devices  and 
control  equipment. 

In  particular,  we  interviewed  representatives  of  the  following 
organizations:  the  National  Association  of  Flight  Instructors;  the  National 
Air  Transport  Association;  the  University  Aviation  Association;  the 
Airplane  Owners  and  Pilots  Association  and  its  affiliated  Air  Safety 
Foundation;  two  flight  schools  in  the  Chicago  area;  two  flight  schools  in 
the  Washington,  D.C.,  area;  and  the  chief  flight  instructors  of  the 
University  of  Illinois  and  Embry-Riddle  Aeronautical  University  at  Daytona 
Beach,  and  other  selected  flight  instructors.  Among  the  equipment 
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manufacturers  and  distributors  we  interviewed  were  Aviation  Supplies  & 
Academics,  Inc.;  Aviation  Teachware  Technologies;  Jeppesen  Sanderson; 
and  Frasca  International,  Inc.  Among  the  academic  experts  we  contacted, 
in  addition  to  our  advisory  panel,  were  the  authors  of  both  the  University 
of  Illinois  and  Embry-Riddle  Aeronautical  University  studies.  At  faa,  we 
spoke  to  officials  in  Flight  Standards;  the  National  Simulator  Center 
(Atlanta);  the  Civil  Aeromedical  Institute  (Oklahoma  City);  Human  Factors 
technical  scientists  (Washington,  D.C.);  and  faa’s  Flight  Standards  District 
Office  (West  Chicago,  Illinois).  We  also  spoke  to  representatives  of  the 
U.S.  Air  Force,  the  U.S.  Navy,  the  National  Transportation  Safety  Board, 
and  the  National  Aeronautics  and  Space  Administration  (Ames, 

California). 

To  identify  other  studies  relevant  to  faa’s  decision,  we  obtained  articles 
cited  in  recent  studies.  We  also  conducted  additional  literature  searches  of 
the  collections  at  two  major  aviation  libraries  and  a  computerized 
literature  search  using  several  bibliographic  databases  and  including  the 
following  keywords  and  phrases:  personal  computer  aviation  training 
device,  pcatd,  accident/incident,  flight  training  device,  ground  training 
device,  part  task  trainers,  aviation  safety,  pilot  training,  air  safety.10  Our 
computerized  literature  search  identified  about  700  studies  and  articles.  Of 
these,  we  selected  68  dealing  with  personal  computers,  simulation  issues, 
or  aviation  safety,  including  all  the  citations  that  appeared  to  involve, 
discuss,  or  present  empirical  research  in  these  areas.  Thirty-nine  abstracts 
were  further  selected  as  relevant  enough  to  obtain  full  text  copies, 
including  all  those  that  appeared  to  involve  empirical  research  on  personal 
computer-based  flight  training.  From  all  our  sources,  including  electronic 
and  nonelectronic  literature  searches,  we  found  eight  studies  involving 
computer-based  device  effectiveness,  and  four  of  these  met  all  of  the 
following  criteria:  (1)  examined  computer-based  flight  simulation 
effectiveness;  (2)  compared  computer-based  device  training  to 
airplane-only  training  performance  or  to  training  with  another  approved 
training  device;  (3)  had  been  conducted  using  an  experimental  design  or 
survey,  including  random  assignment;  and  (4)  used  data  from  an 
experiment  not  summarized  elsewhere  in  our  report  (i.e.,  the  two  studies 
cited  by  faa  in  its  decision — one  at  Embry-Riddle  Aeronautical  University 
and  one  at  the  University  of  Illinois).  In  addition,  we  reviewed  studies  and 


10The  databases  searched  included  the  National  Technical  Information  Service  and  Transportation 
Research  Information  Services;  DIALOG,  from  which  the  following  databases  were  searched: 
Dissertation  Abstracts  Online,  GPO  Monthly  Catalog,  PsychINFO,  IAC  Aerospace/Defense  Markets, 
Aerospace  Database,  McGraw-Hill  Publications,  Periodical  Abstracts  Plus  text,  DIALOG  Defense 
Newsletters,  Business  Dateline,  IAC  Newsletter  Database;  and  the  NEXIS  database. 


Page  23 


GAO/RCED-99-143  Personal  Computer-Based  Aviation  Training  Device 


Appendix  I 

Scope  and  Methodology 


articles  on  general  aviation  flight  safety  that  did  not  directly  mention 
either  computer-based  simulation  or  training. 

Many  articles  in  this  field  are  published  in  journals  with  limited  circulation 
and  in  the  proceedings  of  civilian  and  military  conferences.  Though  these 
can  be  difficult  to  identify  from  conventional  bibliographic  databases,  we 
attempted  through  these  combined  methods  to  obtain  all  those  studies 
examining  computer-based  flight  training  that  used  empirical  methods. 

We  conducted  our  review  from  June  1998  through  June  1999  in 
accordance  with  generally  accepted  government  auditing  standards. 
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This  appendix  summarizes  the  studies  by  the  University  of  Illinois  and  the 
Embry-Riddle  Aeronautical  University. 


University  of  Illinois 
Study 


In  the  University  of  Illinois  study,11  researchers  compared  the  performance 
of  students  who  received  some  training  on  a  pcatd  before  their  training  in 
an  airplane  with  the  performance  of  a  group  of  students  trained  entirely  in 
an  airplane.  University  of  Illinois  aviation  students  who  were  already 
private  pilots  and  had  no  previous  instrument  coursework  were  randomly 
assigned  to  one  of  two  groups  for  their  instrument  flight-training  course. 
After  this  study’s  coursework,  passing  students  took  faa  check  rides  to 
attempt  to  obtain  their  instrument  rating.  Performance  measures  were 
available  for  74  students;  roughly  half  the  students  were  in  each  group. 
Students  were  tested  to  demonstrate  proficiency,  and  the  time  and  number 
of  trials  were  recorded  for  each  student.  The  computer-based  device  used 
was  Model  fs-100  by  MDM,  a  model  not  currently  approved  for  use  as  a 
pcatd  by  faa  toward  flight  training  credit  hours;  the  airplane  used  was 
Beechcraft  Sports/Sundowner,  which  has  a  single  engine,  a  fixed-pitch 
propeller,  and  fixed  landing  gear. 

Students’  performance  in  the  airplane  was  not  substantially  different  for 
those  trained  partly  using  a  computer-based  device  and  those  trained 
solely  in  an  airplane.  However,  the  data  showed  small,  but  statistically 
significant,  savings  in  the  total  airplane  time  needed  to  reach  proficiency 
in  the  airplane  by  students  trained  on  the  pcatd  device.  Students  using  this 
device  spent,  on  average,  3.9  fewer  hours  in  the  airplane  than  those  who 
took  their  entire  training  in  the  airplane.  Airplane-only  students  averaged 
51.3  total  hours  to  complete  the  course,  while  students  using  the  pcatd 
averaged  47.4  hours  in  an  airplane  and  26.5  on  the  device. 

This  study  was  not  designed  to  determine  the  amount  of  loggable  time  faa 
should  grant  for  pcatd  use,  according  to  its  authors.  It  did  not  include  a 
student  group  trained  on  a  currently  approved  flight  training  device.  Such 
a  comparison  group  is  of  interest  because  the  10  hours  currently  permitted 
for  pcatd  instrument  training  may  be  substituted  for  part  of  the  15  to  20 
hours  that  can  be  earned  using  a  flight  training  device.  The  study’s  authors 
believe  that  in  an  actual  instrument  course,  where  experimental  controls 
would  not  require  the  use  of  the  pcatd  for  all  course  lessons,  the  3.9  hour 
savings  of  time  in  the  airplane  could  be  achieved  with  fewer  pcatd  hours 


“Henry  L  Taylor.  Gavan  Lintern,  Charles  L.  Hulin,  Donald  Talleau,  Tom  Emanuel,  and  Sybil  Phillips; 
“Transfer  of  Training  Effectiveness  of  Personal  Computer-Based  Aviation  Training  Devices,” 
University  of  Illinois;  November  1996,  Final  Technical  Report  ARL-96-3-/F  AA-96-2,  prepared  for  the 
Federal  Aviation  Administration,  Oklahoma  City,  OK,  Contract  DTFA-94-G-044 
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by  targeting  the  use  of  the  device  to  those  early  course  lessons  where 
training  transfer  is  greatest. 

The  study  does  not  support  a  conclusion  that  a  “negative  transfer” — or 
learning  interference  caused  by  using  pcatds — occurred  when  the  students 
trained  on  the  personal  computer  device  switched  to  the  airplane.  In  the 
few  instances  in  which  this  group  of  students  needed  more  trials  to 
demonstrate  proficiency  in  the  airplane  than  the  group  without  pcatd 
training,  the  differences  were  so  small  that  they  were  not  statistically 
significant.  Differences  of  the  magnitude  observed  could  occur  even  if  the 
two  groups  of  students  had  been  given  exactly  the  same  training.  While  the 
study  does  not  support  a  conclusion  that  negative  transfer  occurred  on 
some  tasks,  it  also  does  not  rule  out  the  possibility  that  it  did  occur.  Small 
amounts  of  negative  transfer  in  an  individual  task  could  have  existed 
without  being  detected  by  statistical  tests. 


Embry-Riddle 
Aeronautical 
University  Study 


The  Embry-Riddle  study  measured  the  flight  performance  of  students 
trained  using  pcatds  and  compared  their  performance  with  that  of 
students  trained  using  a  previously  approved  flight  training  device.12  All 
the  students  had  previously  taken  navigation  and  instrument  coursework. 
Embry-Riddle  students  who  volunteered  for  the  study  were  randomly 
assigned  to  receive  ground-based  training  using  one  of  two  pcatds  or  using 
the  flight  training  devices.  Students  used  the  assigned  ground-training 
devices  during  the  first  part  of  their  training  and  continued  their 
instrument  flight  training  in  an  airplane.  Instructors  were  randomly 
assigned  to  students  and  taught  students  in  each  of  the  three  groups.  In  the 
airplane,  the  performances  of  the  three  groups  of  students  were  measured 
by  the  total  amount  of  time  and  the  total  number  of  trials  students 
required  to  demonstrate  proficiency  in  each  of  eight  maneuvers. 
Performance  measures  were  available  for  a  total  of  66  students.  The 
pcatds  used  were  the  Elite  (20  students  finished)  and  the  IFT  Pro  5.1  (26 
students  finished);  both  devices  had  15-inch  screens,  486/66MHz 
computers,  and  similar  control  features.  The  flight  training  device  used 
was  the  Frasca  141  (20  students  finished).  The  airplane  used  was  a 
Mooney  20J  single-engine  plane. 

The  study  provided  a  preliminary  indication  that  there  may  not  be  very 
large  differences  in  the  ground  flight  training  provided  by  the  approved 


12Steven  Hampton,  William  Moroney,  Tom  Kirton,  and  David  W.  Biers,  “The  Use  of  Personal 
Computer-Based  Training  Devices  in  Teaching  Instrument  Flying,”  Embry-Riddle  Aeronautical 
University,  June  15, 1994.  Prepared  for  the  Federal  Aviation  Administration,  CAAR  15471931,  Grant  No. 
92-G-0015. 
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flight  training  devices  and  the  two  pcatds  studied.  No  statistically 
significant  differences  were  found  in  the  airplane  performance  of  the 
groups  trained  on  the  three  devices,  as  measured  by  trials  and  hours  to 
meet  faa’s  established  practical  test  standards. 

The  study  did  not  and  was  not  intended  to  address  the  issue  of  how  many 
hours  of  training  on  pcatds  faa  should  grant  for  an  instrument  rating, 
according  to  one  of  the  study’s  authors.  The  study  also  did  not  show  how 
much,  if  any,  training  received  on  each  of  the  ground-based  devices  was 
transferred  to  the  airplane.  The  performance  of  a  group  of  students  who 
received  all  their  training  in  an  airplane  would  be  required  before  an 
estimate  of  how  much  less  time  in  an  airplane  students  receiving  training 
on  a  pcatd  or  flight  training  device  might  need.  Because  no  empirical 
studies  have  been  conducted  on  the  effectiveness  of  training  on  a  flight 
training  device  compared  with  airplane-only  training  for  instrument 
students,  this  study’s  comparative  data  cannot  be  used  to  conclude  that 
pcatds  provide  effective  instrument  training. 
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We  identified  a  number  of  other  issues  from  interviews  with  safety, 
industry,  and  academic  experts  that  could  affect  the  long-term  use  of 
computer-based  devices.  These  issues  are  the  lack  of  the  devices’  physical 
similarity  to  actual  airplanes  and  the  potential  for  associated  problems 
from  negative  learning  from  simulation — that  is,  a  pilot’s  reverting  to 
incorrectly  learned  behaviors  in  an  emergency  or  under  stress — and  the 
possibility  that  the  smaller-sized  instruments  on  the  devices  could 
interfere  with  a  pilot’s  instrument  scan.  Possible  benefits  from  pcatds 
include  their  availability  and  use  in  teaching  concepts  and  procedures, 
such  as  the  flexibility  they  provide  by  permitting  a  pilot  to  practice  on  a 
particular  type  of  airplane,  to  see  a  graphic  display  of  performance,  and  to 
fly  under  varying  scenarios  of  instrument  failure,  system  failure,  and 
weather  conditions. 

Several  simulation  experts  believe  that  credit  hours  should  not  be  granted 
for  pcatd  use,  that  the  two  major  studies  do  not  show  the  training 
effectiveness  of  either  pcatds  or  flight  training  devices,  and  that  there  is  no 
evidence  that  one  hour  in  any  training  device  equals  one  hour  in  an 
airplane,  particularly  for  the  older,  grandfathered  flight  training  devices. 


Views  on  the  Lack  of 
Physical  Similarity 
Between  PCATDs  and 
Aircraft  Instrument 
Panels  and  Controls 


One  of  the  main  concerns  expressed  by  those  opposed  to  granting  credit 
for  instrument  training  time  on  pcatds  is  that  pcatds  do  not  have  sufficient 
similarity,  or  physical  fidelity,  to  an  airplane’s  instrument  panel  and 
controls.13  For  example,  pcatds  have  display  screens  showing  instruments 
that  are  smaller  than  they  actually  are  in  the  cockpit,  and  the  placement  of 
some  pcatd  switches  and  dials  represented  on  the  display  screen  are  in  a 
physically  different  position  than  in  the  aircraft— alongside  or  below  a 
desktop  computer  monitor.  Flight  training  devices,  in  contrast,  can  include 
more  instruments — often  actual  aircraft  instruments — that  can  be 
mounted  on  a  dashboard  panel  larger  than  a  computer  screen,  permitting 
greater  physical  similarity  to  an  aircraft.  On  the  other  hand,  flight  training 
devices  represent  a  standard  aircraft  configuration,  while  pcatd  software 
can  be  easily  changed  to  represent  a  specific  aircraft.  Figure  111.1  shows  an 
approved  flight  training  device. 


13The  literature  discusses  at  least  three  different  kinds  of  fidelity,  including  physical  fidelity— whether 
the  simulation  “looks  right;”  functional  fidelity— whether  the  simulation  “acts  right;”  and 
psychological  fidelity— whether  the  simulation  “feels  right.” 
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Figure  111.1:  Approved  Flight  Training 
Device 


Note:  This  is  a  previously  approved  flight  training  device  with  an  optional  visual  system  (Frasca 
131). 

Source:  Frasca  International. 
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The  potential  impact  of  these  differences  between  pcatds  and  flight 
training  devices  on  pilot  training  is  not  clear.  Some  experts  and 
representatives  of  the  flight  training  device  industry  believe  that  devices 
that  are  less  similar  to  the  airplane  and  have  lower  physical  fidelity  could 
interfere  with  learning.  Some  simulation  experts  believe  that  the  more  the 
simulator  looks,  feels,  and  “flies”  like  the  airplane,  the  greater  its  training 
value  will  be,  particularly  for  experienced  pilots.  However,  other  experts 
and  many  research  studies  suggest  that  high  physical  fidelity  is  not 
essential  for  a  device  to  have  training  value;  that  students  using 
low-fidelity  devices  often  perform  as  well  as  those  using  more  complex, 
higher-fidelity  devices;  and  that  the  effectiveness  of  a  training  device 
depends  on  how  the  device  is  used  in  a  training  curriculum.14 

Negative  transfer — learning  that  can  interfere  with  the  performance  of  a 
task  rather  than  improve  it — can  occur  when  a  familiar  instrument  or  knob 
is  placed  in  a  dramatically  different  position  in  a  training  device  than  in  an 
aircraft.  One  expert  on  aviation  psychology  told  us  that  negative  transfer 
from  pcatds  would  be  unlikely  but  possible.  A  pilot’s  switching  from  a 
training  device  to  an  airplane,  or  from  one  airplane  to  another,  according 
to  interviews  with  aviation  psychologists,  would  rarely  involve  locating 
control  yokes,  knobs,  or  switches  that  are  completely  reversed.  According 
to  one  expert,  however,  if  a  pilot  trains  in  a  plane  with  the  retractable 
landing  gear  switch  on  the  left  side  of  the  cockpit  and  the  flap  switch  on 
the  right  and  then  flies  another  plane  with  the  landing  gear  switch  on  the 
right  and  the  flap  switch  on  the  left,  negative  transfer  of  learning  could 
occur  if  a  pilot  were  to  confuse  the  two  switches. 

In  general,  the  academic,  private,  military,  and  airline  flight  training 
experts  we  interviewed  did  not  believe  that  the  issue  of  pcatds’  similarity 
to  the  airplane  presented  safety  problems.  Several  safety  experts  noted 
that  previously  approved  flight  training  devices  do  not  typically  represent 
one  particular  type  of  aircraft  and  often  include  instrument  panels  that 
differ  from  those  on  the  plane  in  which  the  student  will  train  and  later  fly; 
students  will  have  to  adjust  from  the  training  device  to  a  somewhat 
different  airplane.  While  familiarity  with  a  particular  type  of  plane  is  an 
important  safety  consideration,  few  students  will  fly  only  one  type  of  plane 
and  therefore  will  have  to  adjust,  using  training  devices,  to  different 


14Gavan  Lintern,  S.N.  Roscoe,  J.M.  Koonce,  and  L.D.  Segal,  "Transfer  of  Landing  Skill  in  Beginning 
Flight  Training,”  Human  Factors  32,  (1990),  pp.  319-327;  Gavan  Lintern,  Henry  L.  Taylor,  Jefferson  M. 
Koonce,  and  Donald  Talleur,  “An  Incremental  Transfer  Study  of  Scene  Detail  and  Field  of  View  Effects 
on  Beginning  Flight  Training,”  Proceedings  of  the  Eighth  International  Symposium  on  Aviation 
Psychology,  Columbus,  Ohio,  1995,  p.  737, 
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instrument  displays,  a  different  feel  to  the  controls,  and  different 
performance  characteristics. 

Another  potential  danger  could  occur  when  a  pilot,  in  an  emergency, 
automatically  reverts  to  incorrectly  learned  responses,  skills,  or 
procedures.  This  stressful  reversion,  according  to  some  industry 
representatives,  could  be  the  result  of  the  lower  physical  fidelity  of  pcatds. 
Other  experts  dispute  this,  noting  that  instrument  students  are  already 
pilots  licensed  to  fly  in  good  visibility  conditions  and  already  have  some 
experience  using  airplane  instrumentation;  one  expert  noted  that  incorrect 
and  dangerous  initial  instruction  can  take  place  in  an  airplane  as  well  as 
while  using  a  pcatd.  Another  expert  said  that  quality  instruction  is  more 
important  than  the  type  of  training  device,  but  this  type  of  problem  would 
be  difficult  to  detect. 

Several  experts  were  also  concerned  that  the  smaller  size  of  pcatd 
instruments — usually  about  three-fourths  actual  size — coupled  with  the 
smaller  size  of  the  pcatd  computer  monitor  screen,  could  affect  the  quality 
of  a  pilot’s  instrument  scan — a  critical  skill  to  retain  constant  spatial 
orientation  by  using  instruments  alone.15  In  poor  visibility,  a  pilot  must 
quickly  scan  the  correct  instruments  that  are  most  important  to 
completing  various  maneuvers,  as  well  as  to  maintain  level  flight. 
Instrument  scan  may  also  be  impaired  if  pcATD-displayed  instruments 
suffer  from  lower  fidelity  than  flight  training  devices,  some  pcatd  critics 
believe.  Figures  III.2  and  III.3  show  an  actual  instrument  panel  and  the 
pcatd  version  of  that  panel. 


15Spatial  orientation  is  knowing  the  position  of  the  plane  in  space  and  its  speed,  direction,  and  up, 
down,  and  sideways  orientation.  A  blindfolded  passenger  in  a  plane — even  an  experienced 
pilot-quickly  loses  the  ability  to  accurately  sense  whether  a  plane  is  climbing,  descending,  or  turning, 
in  part  because  of  the  confusing  multiple  sensations  of  acceleration,  deceleration,  and  turning  from 
vestibular  and  other  cues.  G-forces  and  even  a  turning  of  the  head  during  a  turn  or  climb  can  cause 
equally  misleading  illusions.  A  pilot  flying  relying  on  visual  cues  can  be  tragically  mistaken  by  subtle 
optical  illusions  under  certain  conditions  that  might  be  prevented  by  the  better  use  of  instruments  to 
confirm  the  plane’s  situation. 
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Figure  111.2:  Instrument  Panel  Photo  of 
Cessna  172 


Source:  The  Cessna  Aircraft  Company. 
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Source:  Aviation  Teachware  Technologies 


While  instrument  and  panel  size  could  be  potential  problems  for  a  pilot’s 
instrument  scan,  experts  responded  to  this  potential  problem  in  several 
ways.  Several  noted  that  a  pilot’s  instrument  scan  must  be  altered  to  read 
different  airplanes’  instrument  panels  and  so  may  be  aided  by  practice  on 
a  pcatd  that  resembles  the  particular  plane  he  or  she  flies  because  primary 
instruments  would  be  correctly  spaced  although  not  full-sized.  A  flight 
instructor  noted  that  larger  computer  monitors  are  becoming  less 
expensive  and  that  larger  screens  can  increase  the  size  of  a  pcatd’s 
instrument  display.  A  safety  expert  noted  that  whether  an  instrument  is  an 
actual  airplane  instrument  or  displayed  on  a  video  screen  is  not  important. 
Other  experts  say  that  clarity  on  a  video  screen  can  be  very  good — most 
commercial  airliners  now  display  instruments  on  video  screens  in  “glass 
cockpits.”  Another  expert  said  that  while  very  few  general  aviation 
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aircraft  now  have  glass  cockpits,  they  will  become  more  common  in  the 
future. 


Views  on  the  Advantages  Experts  we  interviewed  saw  the  main  value  of  pcatds  in  teaching 
of  PCATDs  procedures  and  concepts,  rather  than  the  complete  set  of  skills  needed  to 

fly.  Many  experts  and  flight  instructors  believe  that  isolating  a  single 
procedure  on  a  training  device  can  help  a  student  focus  on  that  lesson  and 
learn  more  effectively,  particularly  when  introduced  to  a  new  topic, 
without  the  distractions  of  simultaneously  flying  the  airplane. 

In  general,  the  experts  we  interviewed  believed  that  the  potential  safety 
advantages  of  pcatds  outweigh  their  potential  risks.  Among  these  possible 
benefits  are  the  flexibility  of  pcatd  software  to  mimic  the  instrument 
configuration  and  performance  of  a  variety  of  airplanes;  permitting  the 
pilot  to  fly  a  half-dozen  approaches  in  an  hour  compared  with  perhaps  one 
in  an  airplane;  allowing  the  pilot  to  see  and  save  a  chart  of  a  flight  as  a 
learning  tool;  and  allowing  the  simulation  of  systems  failures,  specific 
instrument  failures,  and  a  variety  of  weather  conditions,  such  as  wind 
speed  and  level  of  visibility.  Figure  III.4  shows  a  pcatd  diagram  of  a  flight 
path,  figure  III.  5  shows  a  pcatd  screen  to  set  instrument  failures  and  figure 
III.6  shows  a  pcatd  screen  to  set  weather  conditions. 
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Figure  111.5:  Screen  to  Set  Failures 


Source:  Aviation  Supplies  &  Academics,  Inc. 
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Figure  III. 6:  Screen  to  Set  Weather 
Conditions 


Views  on  Granting  Training 
Credit  for  PCATDs 


Source:  Aviation  Teachware  Technologies. 


Some  safety  experts  also  see  the  wider  availability  and  use  of  relatively 
inexpensive  pcatds  as  a  benefit,  since  most  instrument  students  are  not 
likely  to  have  easy  access  to  a  flight  training  device.16  The  wider  use  of 
pcatds  for  continuing  training  could  potentially  improve  the  overall  level 
of  general  aviation  pilots’  instrument  skills  and  judgment,  which  in  turn 
could  improve  pilot  safety. 


According  to  several  experts,  pcatds  have  a  role  in  training  but  that 
students  using  pcatds  should  not  be  granted  credit  hours  for  instrument 
coursework.  These  experts  believe  that  neither  of  the  two  major  studies 
showed  that  pcatds  had  achieved  a  level  of  training  effectiveness  so  that 
one  hour  of  device  training  equals  one  hour  in  the  airplane;  however,  faa 


16According  to  PCATD  manufacturers,  many  PCATDs  are  sold  to  individuals  either  for  practice  at 
home  or  possibily  for  individual  Part  61  instruction. 
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assumes  that  one  hour  in  the  training  device  equals  one  hour  in  the 
airplane.  These  experts  question  whether  the  performances  of  the  pcatd 
students  in  the  Embry-Riddle  Aeronautical  University  and  University  of 
Illinois  studies  were  due  to  the  pcatd,  training  received  in  the  course 
curriculum  or  in  the  airplane  itself  during  the  check  ride.  They  said  that 
the  previously  approved  flight  training  devices  had  also  not  demonstrated 
their  effectiveness.  These  experts  believed  that  both  pcatds  and  flight 
training  devices  should  be  included  under  one  set  of  standards  and  that  a 
device  should  not  be  approved  for  instrument  training  credit  hours  until  it 
shows  that  (1)  one  hour  in  the  device  is  as  effective  for  training  as  one 
hour  in  the  airplane  and  (2)  the  number  of  credit  hours  now  permitted  are 
the  appropriate  amounts  that  should  be  approved  for  instrument  training. 
According  to  these  and  several  academic  flight  training  and  safety  experts, 
pcatds  are  superior  to  the  older,  grandfathered  flight  training  devices  that 
are  still  approved  for  the  full  15  to  20  hours  of  instrument  flight  training  as 
well  as  to  meet  instrument  recency  (currency)  requirements.17  Figure  III.7 
shows  an  older  flight  training  device  that  was  grandfathered  in  for  use  as  a 
flight  training  device. 


17These  older  model  flight  training  devices  may  be  mechanically  driven  desktop  models  that  are  not 
computer-driven. 
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Figure  111.7:  Example  of  Grandfathered 
Flight  Training  Device 


Note:  This  is  an  ATC  610  tabletop  device. 
Source:  Air  Safety  Foundation. 


Despite  anecdotal  evidence  that  there  has  been  little  difference  in  course 
completion  hours  between  instrument  students  who  are  trained  to 
proficiency  in  the  airplane  and  those  trained  partly  using  flight  training 
devices,  we  found  no  research  showing  that  this  level  of  instrument 
training  effectiveness  has  been  documented  either  for  pcatds  or  approved 
flight  training  devices.  However,  a  1971  study  of  beginning  flight  students 
found  that  an  older  training  device,  the  Link  GAT-1,  transferred 
100  percent  of  1 1  hours  of  training  value  to  the  student’s  airplane 
performance,  with  a  training  effectiveness  ratio  of  one-to-one;  in  other 
words,  under  certain  conditions  for  beginning  students,  one  hour  in  a 
flight  training  device  can  save  one  hour  of  time  the  student  would 
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otherwise  need  in  the  airplane  to  reach  proficiency.  In  a  1973  study,  also 
on  beginning  flight  students,  airplane  training  time  savings  appeared  to  be 
greater  than  one-to-one  in  the  first  few  hours  of  training,  where  one  hour 
in  the  device  saved  more  than  one  hour  in  the  airplane  and  appeared  to 
decline  in  effectiveness  to  less  than  one-to-one  as  time  in  the  flight  training 
device  rose  to  11  hours.18 

According  to  one  expert,  the  assumption  of  one-to-one  substitution  of 
training  device  hours  for  airplane  hours,  though  important,  is  secondary  to 
other  concerns  because  few  students  complete  their  instrument 
coursework  in  only  the  minimum  required  35  to  40  hours  but  instead 
average  about  70  total  hours  to  reach  proficiency. 


18H.K.  Povenmire  and  S.N.  Roscoe,  “An  Evaluation  of  Ground-Based  Flight  Trainers  in  Routine 

Primary  Flight  Training.”  Human  Factors  (1971),  13  (2),  pp.  109-116; _ Incremental 

Transfer  effectiveness  of  a  ground-based  general  aviation  trainer.”  Human  Factors  (1973).  15  (6),  pp. 
534-542. 
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